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Introduction 


It has been stated that approximately 50 - 75 % of our brains are comprised of 
neurons devoted to processing visual stimuli. 99% of the American population 
watches television, with a large proportion relying on it as their primary source 
of news. The saying "seeing is believing" attests to the trust we place in our 
visual sensory apparatus. We speak of "eye-witnesses" in legal proceedings, and 
in the accuracy of visual memory. This concept of visual, or "photographic 
memory” is powerful and useful. Having a very high bandwidth (as opposed to 
our ears), the eyes are sending vast amounts of data to the brain every second for 
processing. The method by which we process this information is still largely a 
mystery, but it is clear that "visual thinking" is based upon an inter-relationship 
between visual perception and memory. E.W. Sinnot, the American biologist 
and philosopher, has stated that, "Stored images in the mind are the basis for 
new creative ideas." 


With this in mind, we can begin to look at the history of visual art as a recorded 
history of ideas and knowledge. We know that cave paintings showed us not 
only elegant and aesthetic (even by today's standards) representations of ancient 
concerns, but also that primitive man, to the best of his abilities, possessed a 
functional understanding of the nature of the materials used to make the 
paintings. As Paul Klee has stated, "The artist knows the nature of all his 
materials, for part of his work has consisted in examining them precisely, one 
after the other, in order to master them with intelligence." So, from the cave 
painter up through current time, artists have to some degree also been chemists 
or technologists. In fact, until last few hundred years, art and science were 
closely allied. The relationship between creative thinking and scientific 
advancement has always been close. Specifically, the role of intuition in the 
creation of a theory or hypothesis, the application of a knowledge of geometry, 
in addition to the use of a rigorous method for recording observed experience, 
has formed the basis for many scientific advances over the years. Albert 
Einstein, in "Essays in Science," stated, "Man tries to make for himself in the 
fashion that suits him best, a simplified and intelligible picture of the world; he 
then tries to some extent to substitute this cosmos of his for the world of 
experience, and thus to overcome it. This is what the painter, the poet, the 
speculative philosopher and the natural scientist do, each in his own fashion." 
("Principles of Research" from Essays in Science, 1934) 


In light of Einstein's insight, and in the context of Scientific Visualization, I 
have asked myself three questions: 


1. What is an artist? 
2. How are artists like or not like scientists? 
3. How can artists and scientists work together? 


1. What is an artist? 


An artist is many things, but generally it could be said that artists are individuals 
who are working towards a deeper understanding of the world in and around 
them. Artists are acutely sensitive to the environment, organizers of large 
amounts of data into an aesthetic expression of his or her world. Artists are 
explainers and representers, pattern matchers, people who find unusual 
relationships between events and images. Artists are creative intedisciplinarians. 
They rework and integrate information, subject data to functions and tests, 
render or give the data shape, express essence and make clear. They help us to 
see in new ways and understand that which is difficult to see, making the 
invisible visible. 


Artists are idea people, creators and conceptualizers, predictors, people with 
ideas connected to the future. They are people who create something completely 
original and new, something beyond the known boundaries of the information 
base. By using or inventing new tools, they show new uses and applications that 
synergize and synthesize fields. Artists push the limits of technologies, bringing 
them to previously unattained goals. They see the next steps, make leaps, put 
together elements in new and powerful ways. 


Artists are bridges to the rest of the world. They understand perception and 
know how to communicate information. They know the technologies and 
language of media. Their work is linked to the general population as their 
inventions and ideas translate directly, often seen as improvements in 
understanding and the quality of life, improvements upon less efficient and more 
laborious methods of working, becoming mainstream as society discovers their 
work and catches up. 


2. How are artists like or not like scientists? 
Mutual Respect and Common Background 


A good number of scientists have backgrounds in visual art, having formally 
studied both art history and creative art production. Many have a deep respect 
for art, and rather than studying it formally, attend museums and galleries and 
maintain contacts with colleagues in the art world. Scientists know from these 
associations that the work of an artist is often as difficult and complicated as 
work in any field. They appreciate and respect the vision and the hard work. 
Conversely, many artists, especially in the more technical fields of photography, 


film, video and now computer art, have good science backgrounds, and have a 
reverence for the achievements of science. There is a parallel respect and a fair 
amount of common knowledge and concern. This results from living in the 
contemporary world with widespread exposure to current ideas and technologies 
through communication media and personal computers, and from a good 
education that includes training in both the humanities and the sciences. 


Aesthetics 


Artists as well as scientists work with abstract symbols, representations for 
various realities and working tools. Even the language used by the two groups is 
similar. Scientists working with mathematics frequently describe a particularly 
good explanation or solution as "elegant." Usually, "elegant" means simple and 
original. This has also been said of good art and design. Scientists apply their 
taste in visual aesthetics to their visual displays of data as do artists, as in the 
case of a scientist who carefully selects colors or arranges forms used in his/her 
design of a chip or rendered image. These are aesthetic decisions, signs that the 
scientist is to some degree thinking as does a visual artist. The intellectual 
bridge of abstraction and aesthetic consideration is fundamental to both groups. 


Art and the Scientific Method 


Just as scientists consider aesthetics, artists often use the scientific method in 
their work, consciously and unconsciously. For example, a ceramicist 
developing a new glaze may begin with a hypothesis he/she wishes to test, and 
with the results of the experiments, will continue to test and modify the original 
hypothesis until the results confirm the hypothesis. This process may actually 
lead the artist to the exploration of fundamental technological problems, even 
though the original motivation for the work was otherwise. Eventually, the tests 
yield data which we see in the form of exotic creations, some successful and 
some unsuccessful. The craft or technique developed may be used to bring forth 
statements by the artist, or as in experimental or process art, the tests and/or 
their results may be viewed as aesthetic statements in themselves. Whatever the 
purpose, it is through the artist's methodology of experimentation and creation 
that new visions, new techniques, and new ideas enter the collective knowledge 
base of the culture. 


Once an experiment has proven repeatable, the results of the experiments end up 
influencing the culture in that they become things to be imitated. Techniques 
and ideas are appropriated by the mainstream and become known and constant 
ways of creating images and representing information. This was, and still is, the 
case in computer graphics. Art Durinski, computer artist/animator formerly with 
Information International Inc. (III), Omnibus Computer Graphics, and Toyo- 
Links of Japan, told me that when he was at III in the late seventies, he "made 
lots of flying logos over grids because they were easy to make." That look first 
set forth by Durinski and his colleagues has been imitated not so much because 
it was easy, but because it was synonymous with computer graphics. The same 
is true of today's metallic and ray traced objects. I have recently seen a demo 
reel of a production company in Southern California, wherein the metallic 


spinning logo was actually made by hand using an airbrush. This computer look 
is the current icon of sophistication desired by our visually demanding, high- 
tech society. 


Visual Thinking 


Visual artists usually think visually, working with drawings and creating spacial 
models in their heads which lead directly to finished works. Art schools 
investigate the models from which artists work in depth, studying their goals, 
work processes, and end results in formal critiques. Like the 3-D artist, the 
sculptor, many of the best scientists with whom I have spoken tell me that they 
frequently work from visual geometric models in their minds, and identify 
research problems on the basis of these visual models or paradigms. From the 
paradigm, the analysis and experiment is derived. This ability to pre-visualize a 
potential solution to a problem and to build 3 or 4-D (3-D plus time) conceptual 
models is basic to their scientific process. But, unlike the art schools, science 
schools do not generally teach or analyze this spatial-conceptual skill. Dr. 
Carver Mead, of the California Institute of Technology Computational 
Neurosystems and Computer Science Departments, often works with visual 
models and informed me that the problem in the sciences is that those who do 
visualize take it for granted, and those who do not know about it, do not know 
about it. Dr. Mead has suggested the importance of teaching conceptual-visual 
skills in secondary and university science education, including drawing, 
filmmaking, and related perception and communication courses. He feels that it 
is important to articulate and formalize the role that conceptual visualization 
(creation of visual paradigms) as well as descriptive visualization (display of 
data) skills play in science education and research. He has stated that the lack of 
training in communication skills has led to a slowing of knowledge transfer 
between various disciplines, an important element in scientific research and in 
the development of new branches of science. This is especially relevant to the 
emerging field of Scientific Visualization, wherein visual thinking applies 
directly to many levels of the process. Being a good communicator, he says, is 
an extremely valuable asset for a scientist. 


Clarity and the Imagination 


How do artists and scientists create models and paradigms? How do they come 
up with fresh and new ideas? What is creative thinking? Both groups are 
working with their imaginations and their intuition, the powerful organizing 
force of the brain which is always active, reaching beyond "normal" thinking 
processes. Creative people often use "educated guessing" or inherent trust in 
their "intuition," the natural organizing force which seems indirect, but often is 
more direct than realized. Intuition builds upon stored knowledge and to me 
means "clarity of perception." It seems that the brain naturally forges neural 
pathways linking information in new ways, reorganizing the data, and then 
reaching beyond it, creating something new and logical, perhaps just not so 
obvious.This form of logic may not necessarily be verbal or mathematical, it 
may instead utilize visual thinking techniques, such as geometric pattern 
matching. The path to the idea may be based on the acceptance of string of 


unproven assumptions, so that part of the work of the scientist or artist involves 
exploration of the assumptions. In any case, the combined skills of logical 
(visual, mathematical, or verbal) and free(the ability to work with unproven 
assumptions) thinking can lead to interesting questions and potential 
breakthroughs. 


This idea formation or "insight of clarity" sometimes occurs in one's sleep, and 
at other times when playing or daydreaming. This process is useful for artists in 
identifying a concept or image, and helps researchers of all kinds in identifying 
problems and their solutions. The subconscious mind keeps working as our 
conscious mind goes onto something else, then suddenly, the Eureka! 
experience. Both artists and scientists use their imaginations and logic, both 
consciously and unconsciously, within a given framework on a particular set of 
events or problems. Creative thinking is inherent to all fields of endeavor, since 
it is a human characteristic. The brain appears to synthesize the accumulated 
information and leap to new ideas, new conceptions, new models for 
understanding the world. 


Art is to Design as Science is to Engineering 


In visual art, the database of information with which a creative mind works 
consists to a large degree of visual images, drawn from a myriad of sources. 
Sorting and making sense of this data by the artist is a highly specialized and 
refined task, potentially occurring partly in the retina or optic nerve, and partly 
in the brain. (In a way, an artist is like a very good computer system!) 


While both artists and designers have large stored databases of images, it is 
important to consider the difference between the two. Designers usually work as 
rearrangers of existing data, applying it to an existing model. The fine artist, 
however, works with and beyond repetition of existing patterns, if necessary 
synthesizing wholly new modes of representing the data. The fine artist works 
from his/her own thinking process, and is capable of creating original ideas and 
models, forging new relationships beyond the known, stored patterns. A good 
designer can also work with the spirit of risk-taking, exploring unusual options. 
Normally, however, this is not the case, as most designers are strongly 
discouraged from exploring unconventional approaches to design while 
students. This attitude typically goes with them into the professional world. 
Ironically, the winners of design competitions are those who break out of the 
rigid visual molds, those working more in the realm of fine artist. Hence, it is 
truly the spirit of the fine artist that makes the greatest contributions to our 
visual environment. An individual considering new information, new problems 
with new data, synthesizing it and reaching towards the new and the fresh, 
towards clarity and refinement, is either a fine artist or designer working as a 
fine artist. This person brings a fresh perspective to the field and may be able to 
create new conceptual or visual models which better fit the new problems. 


Art is to design as science is to engineering. An artist, like a scientist, is 
involved with the imagination in the conceptualization stage of a problem, 
whereas a designer and an engineer is usually executing the application of art 


and science, usually in repetition of what has been done before. This is not to 
belittle the work of the engineer and designer, who is grounded in the pragmatic 
application of known principles. It is just that the roles are very different. If one 
is discussing the research environment, the focus is usually upon the future and 
on gaining a deeper understanding of the world, and hence, a fine artist is more 
sympathetic to the purpose of the scientist than is a designer. 


3. How can artists and scientists work together? 


Unless the scientist is also an artist, it seems that the very best situation in a 
scientific environment is for scientists to work with a fine artist in a team. This 
artist should be a person with enough science background to understand the 
basis of the research, and potentially a person whose own work is related to the 
research field. This person can help to identify significant problem areas and 
keep a stimulating dialog with the scientists. This person should also be 
sensitive to the purpose of the research, as well as applications to society, a 
thinker with a conscience. This is very important, for such a person could give 
very valuable feedback about long term implications, which could very well 
impact the next level of research undertaken. This artist should also be able to 
work as a designer, applying their sense of the pragmatic to original, fresh 
representations, utilizing their well developed visual perception and 
communication skills in the final display of the data. As in any field, care should 
be taken in interviewing a new member of a research team, to make sure that the 
personalities are a good match. 


I have broken down the research process into 5 parts, and have indicated with an 
* where I feel artists "plug in" well: 


1. * Identification of research problems, conceptualization of visual 
models, especially if research is related to vision and visual perception, 
image processing, image making and control of space. An artist with 
some mathematics and science background, whose own work relates to 
the research, may have good insights and may be very helpful and 
stimulating. 


Description of the experiment 


Gis oN 


* Running of the experiment, gathering data, testing, and modeling for 
study (very important). In the case of tool development and evaluation, 
artists are great testers, they push a medium to its limits, working very 
hard and long hours. It has been said that "If there is a bug, they will find 
it." Artists spend a great deal of time making and refining the data, where 
scientists spend more time making the tools. 


7. Studying of results and drawing conclusions; alteration of experiment, 
then go on to next problem (go to 1.) 


9. * Dissemination, giving form to the display of the results, for 
communication purposes. Artists plug in here very well. This is because 
of their fluency with visual communication skills and media. For 
example, an animator knows how to pace changes in time, what 
movements and positions in space "read" well. A painter knows how 
brightness and color saturation affect apparent spacial location, and how 
the eye and brain responds to juxtaposition of form and color. An 
architect or sculptor knows how spatial constructs will look from 
different vantage points, even before the representation is made, 
speeding up the visual selection process as well as improving 
understanding of a final design, and providing more aesthetic results. 


Artists "plug" directly into 3 of the items listed above, including (1) 
conceptualization, (3) data gathering, modeling and (5) dissemination, or 
designing the display of the information for communication. While typically, 
artists are thought to be useful in the last stage only, and though I feel that this 
application of the artist to Visualization is important, it is a limited use of the 
artist, missing out on the full benefit of interchange between disciplines. There 
exist excellent models from industry and education to support the integration of 
artists into stages 1. and 3., conceptualization and testing, as well as 
dissemination. I would even argue that in Scientific Visualization, the most 
important may be the first stage. 


Sample Models from Industry and Education 


There exist many excellent examples of research environments which make use 
of artists, exploring their ideas and integrating them into the process. The 
examples I am listing below in this and in the section called Historical Models 
in Related Media, are but a fraction of those in existence, but will serve as 
models to make my point. Often, it seems as though the use of artists is a kind of 
trade secret! Apple Computer is known to make use of artists, especially 
conceptual artists, in their user interface design and research groups. The Apple 
MacIntosh computer is very popular, and it is likely that the artists had a great 
deal to do with the success of the Mac. The use of black type on a white field, 
which we take for granted, is actually unusual for a graphics system. Most 
computer displays are white on black, which is more difficult to look at. Using 
the printed page is an excellent model, probably the suggestion of an artist. 


Pacific Data Images (PDI), a major computer animation company which is 
known for its continuingly fresh ideas and innovative look, has hired artists and 
animators for various production tasks, but expects that they spend 
approximately 25% of their time on-the-job on their own work. It appears that 
PDI understands that the work the artists do on their own, feeds back to the 
work they do for the company. 


Examples from Caltech 


Computer scientist, Brian Von Herzen of California Institute of Technology's 
Computer Science Graphics Group, has stated that access to art and artists has 


helped him in his own computer graphic work. He has become more aware of 
aesthetic considerations such as color, from looking at art and from studying his 
own perceptions. He incorporates his, by now, very developed sensitivity to 
color into his work. In addition, he tells me that while he can compute a scene 
which, according to the algorithm, should model nature correctly, he calibrates 
his creation with an artist's eyes. The artist can tell very quickly if the scene in 
fact is "correct" and looks like reality, since the artist has spent so much time 
studying the visual world. 


Dr. Dan Whelan, who received his Ph.D. in Computer Science from Caltech in 
1985, wrote a rendering program which I used for computing a sequence of 
images during the same year. There was a serious problem which I discussed 
with him: there were no shadows in the scene, and it just didn't look right. If the 
program couldn't compute the shadows, then I would add them with the paint 
program, which I did. Over the following months, Dr. Whelan developed a 
shadow algorithm, which ended up being a significant portion of his Ph.D. 
thesis. Perhaps he would have written it anyway, but my feedback surely helped 
to accelerate this development. 


Misha Mahowald, graduate student in Computational Neurosystems at Caltech, 
has designed a synthetic retina which makes use of the hexagon as a design 
element. She came up with the idea for that structure from looking at the work 
of M.C. Escher. Ms. Mahowald has stated that "artists are windows to 
understanding perception." She and I keep an active dialog about the 
relationship between art and science, especially interactive computer graphics 
and music as it relates to research in understanding and computer modeling of 
vision and hearing. 


Professor Alan Barr, also of the Caltech Computer Science Graphics Group, 
works with the simulation of natural phenomena. He once worked as a graphic 
designer, and this work, he says, has helped him a great deal in his current work. 
His concern for aesthetics is evidenced by his careful and elegant selections of 
color, composition, and form throughout his visual work. His needs for more 
control over the design and structure of time in animated sequences, has led him 
to some of his current research objectives. His work in dynamic constraints 
developed largely from this need for aesthetic control. In addition, Dr. Barr sees 
the benefits of studying the working process of such artists as character 
animators, in order to understand how artists control form for human expression. 
Dr. Barr has been a major force in teaching his students concepts of visual 
aesthetics as well serving in his formal role as a Professor of Computer Science. 


The University of Illinois at Urbana-Champaign 


The University of Illinois at Urbana-Champaign Center for Supercomputing, is 
currently setting an excellent example through joint efforts between artists and 
scientists. Donna Cox, an artist working at the Center, has spoken about this 
fruitful interaction, what she calls Renaissance Teams. These teams make use of 
specialists in art and science, where neither person has to "backtrack" and learn 
the other field in order to work. This is an excellent prototype for the future of 


Scientific Visualization, an approach which serves both the mature scientist and 
artist. The artist focuses on difficult visual decisions, while the scientist focuses 
on the difficult scientific problems. It helps, of course, that Ms. Cox has a 
lengthy experience with science, so that her aesthetic decisions are made with an 
intelligent approach to the research. The interaction of specialists also serves as 
an education of one about the other, thus enriching the two on several levels. 


My Work with Dr. Weinberg at USC 


Among my efforts to stimulate communication and collaboration between artists 
and scientists, I have worked with Dr. Richard Weinberg of the University of 
Southern California (USC) on the creation of an animated sequence depicting 
two models of the Solar System, the Platonic and Ptolemaic, for the Omnimax 
film, "The Seasons," directed by Ben Shedd. I designed the databases and 
animation, and Dr.Weinberg converted the data into a format for display in high 
resolution for filming in 70mm film format. Through my own rediscovery of the 
models of solar system and their relationships, I arrived at a 12 day year, each 
day spanning 30 degrees of a 360 degree rotation about the sun. In working with 
Dr. Weinberg, we separated the tasks of art/design and computation, so that we 
each focused on our strengths. Support for this film came from the Science 
Museum of Minnesota, among others. 


Historical Models in related Media: 
John Whitney, Sr. 


John Whitney, Sr., considered the father of computer graphics and animation, 
was an experimental filmmaker who came to the use of computers from his need 
to control graphic artwork below a film camera. He used a World War II analog 
computer, an M-5 Antiaircraft Gun Director, augmented later by an M-7, and 
hybridized it to make the first motion-control films. As Gene Youngblood writes 
in Expanded Cinema (1971 - page 208), "The analogue computer work gained 
Whitney a worldwide reputation, and in the spring of 1966 International 
Business Machines became the first major corporation to take an "artist in 
residence" to explore the aesthetic potentials of computer graphics. IBM 
awarded Whitney a continuing grant that has resulted in several significant 
developments in the area of cybernetic art." 


Computer Art at Bell Labs 


Kenneth Knowlton, computer scientist working in the sixties and seventies at 
Bell Labs in Murray Hill, New Jersey, collaborated with a number of artists, 
including Stan Vanderbeek and Lillian Schwartz. Vanderbeek worked with 
Knowlton on a number of films, including Poem Fields, "whose micro-patterns 
seem to permutate in a constant process of metamorphosis which could very 
likely include a hundred thousand miniscule changes every second." 


Gene Youngblood in Expanded Cinema (page 248) interviewed Vanderbeek 
about his work with Knowlton at Bell Labs: 


"The present state of design of graphics display systems," Vanderbeek explains, 
"is to integrate small points of light turned on or off at high speeds. A picture is 
‘resolved' from the mosaic points of light." The artist seems to feel that this 
process bears some physiognomic similarities to human perception. "The eye," 
he notes, "is a mosaic of rods and cones." ' 


Other scientists, such as B. Julescz and C. Bosche, made very interesting and 
aesthetic works as a result of experiments in the psychology of perception and 
human vision. The works of these and other collaborative efforts over the past 
two decades has led directly to the development of our current computer 
imaging and display technologies and applications. 


Bell Labs Sound and Speech Synthesis Research 


While Bell Labs is famous for its computer graphics artists, it is also famous for 
its Composer-in-Residence program, in existence for over two decades. The 
Composer-in-Residence Program, as headed by Dr. Max Mathews, was set up in 
the early 60's in an effort to make computers available for cultural use, as well 
as an aid in the research effort. As Dr. Mathews, in Computer Composers -- 
Comments and Case Histories stated, 


"Encouragement was offered from a feeling of responsibility for bringing new 
technology to the attention of those in the cultural community who can make use 
of the new tools, and partly because increased understanding of sounds may 
improve speech communication. Composers are very creative and have very 
good ears for making and analyzing sounds. Much more could be said about the 
impact of art on technology..." 


In my interview with Dr. Mathews, now at Stanford University, I asked, "Did 
the composers help to accelerate the development of the sound and speech 
synthesis technology? "Yes," said Dr. Mathews. "A good example is Jean- 
Claude Risset," who was both a physicist and composer. Risset made the first 
catalog of sounds with a computer, including brass sounds such as a trumpet, 
flutes, woodwinds, pianos, bells, and "a host of percussion instruments plus 
numerous non-instrumental sounds. In addition," writes Mathews, "he has 
invented another musical dimension--spectral pitch control--which an be used to 
produce tones which seem to be simultaneously ascending and descending in 
pitch. He has become so expert in tone quality that he can achieve a desired 
timbre with almost no experimentation. He has made his expertise available to 
the world by making a catalogue of sounds. I believe that this is the most 
important musical document of the century." This concept of a sound catalog, 
and the models set forth by Dr. Risset, is part of virtually every synthesizer on 
the market today. 


I asked Dr. Mathews if the composers at Bell Labs had to be their own 
technicians and scientists. He said that while many were able to develop their 
own technological devices and tools, this was not required. The Laboratory was 
set up to provide a supportive scientific environment. Since the composers were 
formally invited into the Lab, computer time was set aside for their work, 


though sometimes the hours were odd and in the middle of the night. They were 
invited to make experiments in music as well as to compose pieces. 


Recognition, Documentation, and Formal Integration 


As discussed, artists and scientists have worked together for many, many years, 
but now it is beginning to be a more formal collaboration. The examples of 
current team-work at such places as the University of Illinois Center for 
Supercomputing, imply a movement,a profound change that is now being 
noticed. These formal collaborations are, however, still more the exception than 
the rule. Even though artists have been and continue to be of great influence, 
their positions in research environments are relatively informal. This results 
from a general lack of articulation of the artists' contributions, and lack of 
justification for support of the artist. Therefore, the influence and contribution of 
the artist has been virtually undocumented. Leonardo,an excellent publication on 
the subject of art and science, begins to address this need, but it is not nearly 
enough. And while the work of Ms. Cox at the University of Illinois is an 
excellent model and starting to be documented, much remains elsewhere to be 
recognized and formalized. There is a great need for an honest and expanded 
study of the contributions artists make to science throughout the United States. 
An investigation should be undertaken, a report written and published, and 
formal changes made in the granting and support procedures in agencies, so that 
serious consideration is given to the artist, reflecting this awareness. Arguments 
and justifications for the formal integration of artists into scientific research 
should be made at the highest levels, so that this becomes a standard, not an 
exotic and unusual situation. Changes should be made so that artists are not 
expected to work for free.The institutions that include artists will probably be 
seen as visionary, forward thinking environments, and may in the end have 
better chances for support, because their research may be better. This support 
and reputation would attract the best artists in the field, bringing strong ideas 
and the best energies to the research environment. The formalization of the artist 
in Scientific Visualization in particular, is especially important since the work 
draws so much from the visual artist who is already very much involved. 


Artist-in-Residence Programs 


Not only should there be justification for Artist-Scientist Teams, but there 
should be formal Artist-in-Residence Programs, where an artist is working on 
their own work, as did the composers at Bell Labs. These people should be paid 
like any other researcher, and should have access to funding from the same 
sources as the scientists. Special grants should be made available to artists from 
government agencies, as well as from organizations that benefit from the work, 
such as ACM SIGGRAPH and large computer companies. A portion of the 
funds for science should be designated for the artist's research. Artists should be 
considered equal members of the research environment, and their work should 
be recognized as such. Their work should be seen in the context of the working 
environment, the influence it has on the sciences, and should be published in the 
scientific journals when the content merits it. The artists selected for residencies 


should be carefully screened, so that their capabilities, interests and goals relate 
to the overall goals of the research environment. It is very important, though, 
that the artist be free to pursue their own research ideas, as this will bring an 
increased possibility for advancement of the field. 


Scientist-in-Residence Programs 


Scientists can benefit from immersion in the art environment. They can learn a 
great deal about their own perception and develop excellent communication 
skills. They can work from these skills towards solving problems of not only 
aesthetics and refined visual composition, but also user interface design, and 
new applications as set forth by artists working in various media. For example, 
multi-screen interactive digital-video performance, computer controlled 
sculpture, or the integration of sound and image may stimulate new ideas and 
directions for research. They may be learning more about the psychology of 
perception and the working and thinking processes of artists, so as to make 
better tools for artists, and other users of computers. This could lead to many, 
many new applications and insights. It can also be of great benefit to the art 
environment, helping both the artist and scientist to develop new tools and reach 
new goals. Because this exploration of tools and methods used in, and the 
methods for computer art and animation, the work really is research. But 
because it takes place in the art environment, it is not formally designated as 
such. Therefore, it is my feeling that funds for the work of scientists and artists 
in art environments should be available from organizations that support research 
elsewhere. 


Art Shows 


Just as an individual stimulates the research environment, so does the work of a 
collective group of artists. The value of art shows in general, and computer art 
shows in particular cannot be measured in dollars spent by attendees. There are 
other rewards in life besides money! The artworks can lead us to an improved 
quality of life, suggesting new ideas or improvements in our working methods 
and living environments. The true value of art shows like those at ACM 
SIGGRAPH and NCGA, lies in these intellectual rewards, the seeds they plant 
in the minds of the viewing and participating audience. It may be years later that 
the seeds take root and lead to major research efforts or applications. But 
without the art shows, the seeds cannot be planted. It is of critical importance 
that art shows such as these be supported, for their impact is invaluable, 
measurable only over a long period of time. 


Art shows should not have to be mounted at the expense of the artists (a 
common reality). While some of these artists are also scientists with Ph.D.'s who 
manage to produce their artwork supported by other activities, most artists 
struggle very hard, without an equivalent amount of support. Because of the 
nature of their work and their unusual backgrounds, they may not fit into a 
normally funded slot. Therefore, I feel that it is important to recognize when one 
is "taking" from the work and ideas of creative artists, and do one's best to "give 


back." Through active support for artists and art shows, we can continue this 
important cross-fertilization and help the entire field to grow. 


The Role of Education 


To help the next generation of researchers, we need educational role models, 
institutions leading the way, not afraid to step forward and set an example. We 
need educational programs which actively bring together artists and scientists at 
all levels. We need to rethink and restructure our curricula, requiring math and 
science for all art students, and humanities and art for all science students. We 
should create new courses which specifically discuss the relationship between 
art and technology, and include, for example, a lecture series which brings in 
role models for the students to see and hear. We should embrace the concept of 
hybrid renaissance people, and for older people who are already specialized, 
create courses which foster mutual understanding and collaboration. We should 
help to improve visual literacy on the part of sciences, and technological literacy 
on the part of the humanities. We can make use of existing programs, expanding 
them where necessary. Art schools have already started this process, now it is up 
to science institutions to level the balance. 


Making use of Existing Art Programs 


Art schools have the courses already developed for a good visual training for the 
sciences. Existing art programs within universities are rich and useful resources. 
Instead of re-inventing color theory, composition, and media studies, the science 
schools should be working toward making use of art departments, sending their 
students to classes in art history and art production, and hiring artists from the 
art departments to help with teaching as well as research projects which involve 
visualization. Why is it, though, that so few science departments have made use 
of these resources? As in research, the problem seems to be one of justifying the 
role of art in support of science. Call it "Visualization Engineering" instead of 
"Art," said one scientist. Perhaps this is the solution, finding a new name for the 
technologically literate artist. In any event, I hope that this problem will be 
resolved as an articulated need for visual literacy on the part of the science 
community becomes more and more of an issue. Art History and Visualization 


The change from "computer graphics" to "visualization," what I call the second 
stage of computer imaging, parallels the history of visual art, wherein the desire 
to accurately represent they optical image realistically gave way to surrealism, 
impressionism, and abstraction. This developed as artists looked for a way of 
representing abstract visual concepts and realities beyond the directly 
perceptible. As the level of sophistication of computer visualization increases, 
and as researchers seek to make the invisible visible, one can sense the direct 
application of the history of art to Visualization. By studying both the history of 
art and contemporary art, it is possible to gain insights into the aesthetic and 
conceptual manifestations and uses of images in society. The accumulated 
knowledge of the fine arts can be extremely useful to Scientific Visualization, a 
field which will rely more and more on visual skills and ideas. A study of art 


history can help to gain insights into visual form giving and unique ways of 
solving problems which could enhance the scientific research environment in 
new and unexpected ways. 


Other Art Courses 


Since so much of Scientific Visualization is based on visual communication, a 
formal visual training beyond art history is necessary. If scientists wish to take 
on the work of the artist themselves, they need better skills for drawing, visual 
and temporal composition, experience with film and video analysis and 
production, including animation, as well as 2 and 3 dimensional design. 


From artists and art courses, scientists can learn about the psychology of 
perception and its application to image making. For example, opposite colors on 
the color wheel cause figure-ground disparities and seem to "jump." In an 
enclosed space with little ambient light, darker colors imply a greater distance 
from the viewer than do lighter colors (depth cueing). In a space with a great 
deal of ambient light, such as outdoors in daylight, colors fade into the grey of 
the atmosphere, that is, they lose saturation as a function of distance. Artists 
know this from observation and analysis. 


Scientists can learn about composition and visual balance from a graphic design 
course. As an example, a symmetrical design appears static, whereas a scene set 
assymetrically and at an angle, appears dynamic. Dynamic images are usually 
more interesting to look at. In addition, studying various color models, such as 
the color sphere of Philip Otto Runge (1810), the Albert H. Munsell model 
(1898), and the CIELAB model, which is known to the science community, can 
help in understanding color. A study of the work of such masters as Albers, 
Itten, and Klee can supplement this study and be of immense value in the 
application of color to form and composition. In film animation one can learn 
from the work of Hans Richter, who developed a vocabulary of temporal spatial 
articulation by eliminating form and color, and working only with the movement 
of the screen itself before a film projector. Through an architecture course, one 
can combine a formal training in plane and solid geometry with the study of 
architects such as Frank Lloyd Wright, and begin to understand our relationship 
to the space in which we live and work. In general, art education can help 
scientists to understand their relationship to the visual world and how to use it in 
their work. 


Art Schools and Computers 


Art schools not only teach traditional art history and production, but computer 
graphics and animation as well. This should be seen as no surprise, as artists 
have always worked with the tools of their age. Computers for graphics were not 
readily available to artists before about 1980, and due to the steady decrease in 
cost and increase in power, art schools are now providing courses in computer 
art, programming and mathematics. Where in the past, artists who wanted to 
learn about computers had to enroll in computer science programs (Duane 
Palyka, of New York Institute of Technology is an excellent example), this is 


less the case today. The new generation feels illiterate without training in 
computers, and virtually all art schools worth their salt offer computer graphics 
and programming courses. Perhaps not as demanding as full fledged computer 
science programs, they are forging the way towards an integrated artist-scientist 
training ground. Science institutions, while not yet embracing the arts with the 
same conviction that the art schools embrace technology, are beginning to 
express the need for visual literacy. At some point the two fields will find that 
their curricula overlap, and that they are much better off if they pool their 
students and resources. 


California Institute of the Arts 


More and more scientists, as well as people from other disciplines, are applying 
to the Computer Animation Program at the California Institute of the Arts 
(CalArts). We have students entering this program with backgrounds that 
include biology, computer science, geology, oceanography, physics, chemistry, 
as well as the more traditional art backgrounds of literature, painting, and 
photography. All entering students must have a good portfolio, an original 
vision and a well developed ability to visually express themselves. The artists 
entering the program without much computer experience are strongly 
encouraged to study electronics and programming, in order to gain a deeper 
understanding of the tools they use, so that they may have a larger technical base 
from which to create. Most of these people find an internal motivation for this, 
and therefore are not required to take the courses. Curiously, more students take 
programming and electronics courses when the choice is theirs to make. In the 
case of a computer scientist who is already far along the technical path, I ask, 
"Why does this person want to study at a fine art school?" In the case of one 
computer scientist who recently applied, he answered, " Now that I know the 
techniques of computer graphics, I want to learn about the meaning of images, 
how to compose them and work with motion and structure in time. I want to 
work towards a humanistic application of my knowledge, and to develop my 
knowledge of and skills for communication through images... I want to realize 
my inner vision." 


What we are seeing is the growth of a diverse and dynamic computer art 
program, one which is formed largely by the arts moving closer to the sciences 
as the sciences are moving closer to the arts. It is a field in terrific flux and 
growth, as each new student brings a wealth of educational and personal 
experience to their work and to the new field. Being both highly technical as 
well as highly malleable, the uses of computers in art and animation at CalArts 
borders closely on research, since the possibilities for serious creative work is 
vast. There is a great deal of freedom for development of computer applications 
at CalArts, perhaps more so than in more rigid environments. Though there exist 
very real financial and hence, technical limitations, the intellectual limitations 
are few. Perhaps this freedom is what attracts the unusual multi-disciplinary 
student to computer art programs like that at CalArts. 


CalArts and Caltech 


In addition to my work at CalArts, where I am Director of the Computer 
Animation Laboratory, I hold a position as Visiting Associate in Computer 
Science at Caltech, where I pursue my personal work and try to improve my 
understanding of computing through auditing of classes. I am also active on 
several levels in trying to catalyse increased interaction between the fields of art 
and science. I continue to invite researchers from throughout the field, including 
Caltech and the Jet propulsion Laboratory (JPL), to speak in my classes at 
CalArts. I try to keep an ongoing verbal and visual dialog between CalArts and 
Caltech. A few years ago, I mounted a gallery exhibit and live performance of 
computer art, animation and music called "ArtsTech '85," in order to bring 
together artists and scientists from the CalArts, Caltech, and Jet Propulsion Lab 
(JPL) in a celebration of technology and art. This show received major attention 
in several international journals, including Novum, a European graphic art 
publication. The evolving relationship between CalArts and Caltech has led 
directly to the employment of a number of my students at Caltech and JPL, 
where their duties have included computer aided illustration and design for 
scientific texts, as well as the creation of computer animation for the simulation 
of Galileo, the un-manned space probe to Jupiter. In addition, I have discussed 
the formation of joint educational programs for students at CalArts and Caltech. 
A sample curriculum for institutions which house computer science and art 
departments is described in general, below. 


Sample Curriculum 


A program which would prepare a student for work in the field of Scientific 
Visualization could be twofold, a Master of Fine Arts Degree, as well as Master 
of Science Degree in Visualization. Students would focus in either the arts or the 
sciences, or an extended program could join the two, yielding a double degree. 
Faculty and curricula could be created entirely anew, or through joint 
educational efforts formed from existing programs. In the latter case, an equal 
number of students from both disciplines would be accepted into the program so 
that neither the the sciences nor the humanities departments would bear the 
major burden of resource allocation, as facilities would be shared. Syllabi would 
be geared towards students and their special backgrounds; artists would learn 
applied math, as well as general computer science courses, including 
programming, electronics, Very Large Scale Integration (VLSJ), and algorithms; 
and for the scientists, drawing, film, video, animation, as well as art history and 
aesthetics would be offered. It is very important to have the very best teachers 
for this purpose, as attitudes impacting the future of the field will surely be 
formed through these courses. In addition, new courses in Scientific 
Visualization would be created and taught by experts in the field. 


The following college level courses would be spread out over a 4 year period: 


Art students would take 1 per semester: Science students 
would take 1 per semester: 


* Algebra ** Art History, 
incl. architecture 


* Trigonometry ** Drawing and 


Painting 
* Plane and solid geometry Sculpture 
* Differential and Integral Calculus Photography 
Topology ** Film and Video 
analysis and production 
Biology Animation 
Chemistry ** Graphic Design 
* Physics Electronic and 
Computer Music 
* Electronics Performance Art 
** Programming Languages: ** Interdisciplinary 
Art 
C ** Computer Art 
Lisp ** History of 
Technology in Art 
Prologue * Geometry 


other new languages 

** Algorithms for Graphics 

** VLSI and Simulation 

** Artificial Intelligence and Expert Systems 
Statistics 
Engineering Courses as related to Specialty interest 


* = required if not taken in high school ** = required 
courses 


Special Courses bringing together artists and scientists could include: 


e Principles of Scientific Visualization 

e Directed Research Projects in Visualization, artists working with 
scientists in teams 

e Projects in Applications of Computer Animation to Visualization 

e Developing an Expert System for Visualization 

e User Interface Design 


Summary 


99% of the population is affected by visual language and communications 
media, and while visually sophisticated, most people are, including our 
scientists, relatively unconscious of this visual language being used. Scientific 
visualization leads scientists to the need for an understanding of this visual 
language, as images become the major mode of data representation and 
communication. Since the eyes take in vast amounts of data in parallel, images 
can be thought of as a high-bandwidth data representations. Kinetic, three 
dimensional models can describe complex behaviors which may be impossible 
to understand or communicate any other way. Since so much of our invisible 
natural world is spatial, the computed models which give these phenomena form 
can communicate better and more widely, what before only a few people could 
model in their minds. Great care should be taken to the display of this data, as 
visual decisions can dramatically affect its comprehension. Visual representation 
of information is a skill which anyone can study and learn, either through 
example, or through formal study of the visual arts. 


Visual thinking relates to not just the output stage of science. For many, many 
years, science has made use of visual models or paradigms. Science education 
should reflect this fact, teaching geometry and art in an effort to help the next 
generation of scientists. The integration and application of art to science and 
vice-versa, should be taught at all levels of our educational systems, so that as 
our technology for communication develops, our users are not left behind. An 
organized effort to merge the fields would enhance the efforts and accelerate the 
growth of Scientific Visualization. 


To facilitate and formalize the particularly considerable contribution of the arts 
to science, we should create funded Artist-in-Residence Programs, funded by the 
government and private companies, where artists would work on their own 
research projects, the product of which could integrate very well with overall 
technological research and development. This is based partly on companies such 
as Apple Computer, which employs artists in various capacities, including in 
user interface design teams, and partly on the model of the Bell Laboratories 
sound and speech synthesis research effort. Bell Labs has had a formal 
Composer-in-Residence program for over 25 years, which has led directly to the 
sound and music technology available on the consumer market today. In the 
field of Scientific Visualization, the artist's understanding of visual perception 
and communication, and their contributions to the identification of research 
problems, should be formally recognized and integrated into the scientific 
environment. In addition, I encourage the formation of Artist-Scientist Research 
Teams, where artists would aid in the visual display of data collected from 
scientific research. This is already taking place at the Center for Supercomputing 
at the University of Illinois at Urbana-Champaign. I also support the creation of 
Scientist-in-Residence Programs, where scientists would pursue their work in 
the context of an art school, in order to gain as much insight into visual thinking 
and aesthetics as possible. We should support Computer Art Shows and 
Exhibitions which bring together artists and scientists for a cross-fertilization of 
ideas. Finally, I encourage mutual curriculum development between art and 
science institutions which could help to bring together the two worlds in a 
common endeavor. It would yield a new generation of visually and culturally 
literate scientists and technologically literate artists, and potentially providing us 
with a unified individual with dual capabilities, the truly contemporary 
Renaissance Person, well prepared for the emerging field of Scientific 
Visualization. 


http://visualmusic.org/text/scivil.html 


